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^ Abstract 

Boltzmann's struggle with a derivation of the Second Law of Ther- 
modynamics is sketched. So is his first derivation of the connection be- 
_G tween entropy and probability in 1877. Planck's derivation and quan- 

c/2 turn mechanical modifications of Boltzmann's connection between en- 

• tropy probability are given next. Then Einstein's objections to a 

purely probabilistic rather than a dynamical interpretation of entropy 
,1^ are discussed. Finally, the dynamical basis of the Sinai-Ruelle-Bowen 

, &-f distribution for very chaotic systems is sketched and appears to be an 

example of Einstein's dynamical interpretation of entropy. 



I. Introduction 

00 

CN Boltzmann had a strong disposition for mechanics and his first papers 

were all devoted to purely mechanical derivations of the Second Law of Ther- 
modynamics. The most important one was that based on the 1872 Boltzmann 
equation, i.e. on the dynamics of binary collisions, where he says at the end: 
"One has therefore rigorously proved that, whatever the distribution of the 
kinetic energy at the initial time might have been, it will, after a very long 
^ time, always necessarily approach that found by Maxwell"^. 

He seems to have overlooked entirely that the Stoszzahl Ansatz, i.e. the 
assumption of molecular chaos used in his equation, was a statistical assump- 
tion which had no dynamical basis. 

It is therefore, in my opinion, ironic that perhaps his most famous achieve- 
ment may well have been the relation of 1877 between entropy and probabil- 
ity, which was devoid of any dynamical feature' 2 '. 
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Whenever later dynamical results were obtained, e.g. by Helmholtz, when 
he introduced his monocycles in 1884^, Boltzmann immediately jumped at 
it and extended it to what we now call the origins of ergodic theory, i.e. a 
dynamical theory W. Also his summarizing lectures in the two volume "Lec- 
tures on Gas Theory" ^ mostly discuss dynamical approaches and reference 
to the entropy-probability relation can mainly be found on a few pages of the 
first book in connection with the statistical interpretation of his iJ-function 
(ref.5,I,p.38). 

However, in this lecture I do want to concentrate on his work on entropy 
and probability first and end with a revival of the dynamical approach as pro- 
posed by Einstein and as later used, in my view, in the dynamical approach 
to phase space probabilities in the Sinai-Ruelle-Bowen (SRB) distribution. 
The sudden switch which Boltzmann made from a purely dynamical to a 
purely probabilistic approach, might well have been due to the critical at- 
tacks of many of his colleagues on the Stoszzahl Ansatz, as exemplified by 
Loschmidt's Reversibility Paradox^ and Zermelo's Recurrence Paradoxal. 

II. Boltzmann's original derivation of S ~ logiy 

This was done in a paper of 1887^: "On the relation between the Second 
Law of the Mechanical Theory of Heat and Probability Theory with respect to 
the laws of thermal equilibrium. " I will sketch first the simplified procedure 
Boltzmann follows in Chapter I of this paper. 

The crucial statement here is: "For an ideal gas in thermal equilibrium 
the probability of the number of "complexions" of the system is a maximum." 

Boltzmann introduced the notion of "complexions" as follows: 

a) Assume discrete kinetic energy values of each molecule, which are repre- 
sented in an arithmetic series: 

e, 2e, 3e, ...,£>£, 

where each molecule can only have a finite number, p, of kinetic energies e. 

b) Before each binary collision the total kinetic energy of the two colliding 
molecules is always contained in the above series and "by whatever cause" 
the same is true after the collision. He says: 

"There is no real mechanical system [to which this collision assumption is 
applicable], but the so-defined problem is mathematically much easier to deal 
with and [in addition] it goes over in the problem we want to solve, when the 
kinetic energies of the molecules become continuous and p — > oo ". 
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Assume that the possible kinetic energies of the N molecules are dis- 
tributed in all possible ways at constant total kinetic energy E. Then each 
such distribution of the total kinetic energy over the molecules is called a 
complexion. What is the number P of complexions, where Wj molecules 
possess a kinetic energy je (j = 1,....,N)7 This number P [which Boltz- 
mann calls "the permutability" or "thermodynamic probability"] indicates 
how many complexions correspond to a given molecular distribution or state 
of the system. A distribution can be represented by writing down first as 
many j's as there are molecules with a kinetic energy je (vjj) etc. Obviously 
P = N\/U? =1 Wj\. The most probable distribution is that for which the {wj} 
are such that P = max or H^^Wjl or also \ogH p j=1 Wj\ a minimum. With the 
constraints Z)j=i Wj = N and Y%=i(j£)wj = E and Stirling's approximation, 
Boltzmann finds then for the probability that the kinetic energy of a molecule 
is se: 



with e = jjf, the average kinetic energy of a molecule. 

Boltzmann adds that: "in order to translate the above [derivation] into the 
mechanical theory of heat, in particular into the introduction of differentials 
[when one makes the kinetic energies of the molecules continuous rather than 
discrete], needs still some thought and a not unimportant modification of the 
formulae. " 

This was worked out in the following three chapters of this paper. 

It ultimately leads then to the results that max P yields Maxwell's expo- 
nential kinetic energy distribution as well as to the developments in Chapter 
V, which I will now discuss. 

In Chapter V of this paper Boltzmann discusses the "Relation of the 
Entropy to that quantity, which I have called the [thermodynamic] probability 
distribution (9) He first makes the connection between what he calls the 
degree of permutability Q or log P given by: 



and the entropy. Here f(r, v) is a continuous generalization of the discrete 
Wj used before, giving the number of molecules at the position r with veloc- 
ity v. The maximum of Q under the constraints that iV and E are given, 
leads then again to the exponential Maxwell equilibrium velocity distribu- 
tion. Noting that the degree of permutability Q differs from the logarithm of 



w s ~ - 



exp —se/e 
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the permutability P only by a constant, one has Q = logP+ constant. If the 
gas was initially not in thermal equilibrium and approaches equilibrium, Q 
must reach a maximum VL max . Boltzmann then computes thermodynamically 
the entropy S for an ideal gas with N particles and average kinetic energy 
or temperature T. For reversible processes he finds then: 



As Abraham Pais remarks in his classic book "Subtle is the Lord...." t 7 ': 
"Boltzmann's qualities as an outstanding lecturer are not reflected in his sci- 
entific papers, which are sometimes unduly long, occasionally obscure and of- 
ten dense. Their main conclusions are sometimes tucked away among lengthy 
calculations. " 

The paper I am discussing is a prime example of this description. Only in 
the text towards the end of the paper the following sentence appears referring 
to the previously obtained formula S — J dQ/T = f2 when he says: 

"Now it is known that when in a system of bodies only reversible changes 
occur, the total entropy of all these bodies remains constant. If, however, 
also irreversible processes occur, then the total entropy of the system must 
necessarily grow. ..." 

According to the equation 



the increase in the sum of the degree of permutability of a system dQ equals 
then the increase of its entropy dS. He says: 

"Therefore the degree of permutability Q is a quantity, which in the equi- 
librium state, apart from a constant factor and an addend, is identical with 
the entropy S. " 

That is: S = CiQ + c' 2 = C\ log P + c 2 , but this formula is not in the paper. 

Boltzmann ends this very long [60 pages] paper by remarking that too lit- 
tle is known both experimentally and theoretically about liquids and solids, 
to generalize his relation S — c\ log P + c 2 from ideal gases to liquids or 
solids. He remarks that he "has given earlier arguments that it is likely that 
also for these states of aggregation, thermal equilibrium will be determined 
by a maximum of the quantity Q, which [also] for such systems [would be] 
identical with the entropy S. " 




dQ/T = n = logP + constant. 



dn = d\ogP = dQ/T = dS 
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III. Planck's derivation of Boltzmann's S = C\ log P + c 2 ® 



It was really Planck who made the step from Boltzmann's paper to S — 
k In W + c, where W is written instead of P. Planck bases his discussion of 
the connection between entropy and probability on the universality of both 
the Second Law and the laws of probability "so that it is to be expected that 
the connection between entropy and probability should be very close". 

Hence he makes the following proposition as the foundation of all further 
discussion: "The entropy of a physical system in a definite state depends 
solely on the [thermodynamic] probability W[= P] of this state. " 

Without knowing this probability W , S can be determined as follows. 

Planck's derivation of Boltzmann's S = f(W) 



The probability W, for a system consisting of two entirely independent 
systems with probabilities Wi and W 2) respectively, is: W = WiW 2 . Then 
Si = fiWx) and S 2 = f{W 2 ). Second Law: S = Si + S 2 or f{W 1 W 2 ) = 
f(W) = f{W\) + f{W 2 ). Differentiating both sides of this equation with 
respect to W\ with W 2 = constant and then the resulting equation with 
respect to W 2 at Wi = constant one obtains: 

f{W) + Wf{W) = 

with the solution: f(W) = klogW + constant or S = klogW + c. Thus 
Planck formulated Boltzmann's connection between entropy S and the per- 
mutability P ~ W, in its definitive form: 

S = k\ogW + c 

Planck notices two differences between his formula and Boltzmann's S = 
ci logP + c 2 : 

a) he replaces Boltzmann's macroscopic expression for ci = R/N by a molec- 
ular quantity which he called Boltzmann's constant k. 

b) the additive constant c 2 is undetermined as is the case in the whole of clas- 
sical thermodynamics. Planck assigns a definite value to c 2 , i.e. a definite 
value to S, by using the "hypothesis of quanta" . 

That is, he assumed that in every finite region of phase space the ther- 
modynamic probability has a finite magnitude limited by [the existence of] 
h, Planck's constant, and can not be infinitesimally small like in the classical 
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case. This allows S to be determined free of an arbitrary constant and at 
the same time to connect the classical and quantum mechanical values of S, 
such that S = at T = (Sackur- Tetrode formula)^'. 

To the best of my knowledge neither Boltzmann's nor Planck's derivation 
of S for an ideal gas has ever been generalized to an interacting gas. Results 
for such a gas can be obtained from Gibbs' microcanonical ensemble but that 
is based ultimately on the unproven ergodic hypothesis. 

IV. Einstein's Objection' 10 ' 

1. From 1905-1920 Einstein repeatedly objected to the probabilistic deriva- 
tion of S — klogW + c. 

2. Basis: the thermodynamic probability W to find the system in a certain 
complexion can only be determined dynamically and not be guessed statisti- 
cally by assigning "ad hoc permutabilities" (P) to complexions of the system. 

3. In fact, it is determined by the frequency that a phase space trajectory 
visits a given region of phase space due to its dynamics in phase space. 

Quote of 191Q[ 10b ] 

"Usually W equals the number of complexions. In order to compute W 
[however] one needs a complete (molecular-mechanical) theory of the system. 
Therefore it is dubious that the Boltzmann principle has any meaning without 
a complete molecular-mechanical theory or some other theory which describes 
the elementary [dynamical] processes [of the system]. In equilibrium, the ex- 
pression S = klogW + c, seems [therefore] devoid of [any] content from a 
phenomenological point of view, without giving in addition such an elemen- 
tary theory. " 

V. The SRB distribution M 

Recently a theory has been developed for systems in a non -equilibrium 
stationary state, which are in principle, not restricted to be near equilib- 
rium. This theory uses a probability or measure for "complexions" of the 
system in its phase space following Einstein's dynamical rather than Boltz- 
mann's (probabilistic) Principle, as Einstein called it. This measure is usually 
called the Sinai- Ruelle-Bowen or SRB measure' 11 ', which will be sketched in a 
physical way now. For simplicity, I will use a two dimensional representation. 
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SRB distribution (physically) 



Consider a smooth and very chaotic classical dynamical system (Anosov- 
like). This Chaoticity is based on the hyperbolicity of the points representing 
the system in its phase space. Each such point has two manifolds (cf.fig.l): 

a) an unstable manifold (u), on which two separate points near a given phase 
point exponentially separate from each other; 

b) a stable manifold (s) on which two separated points near a given phase 
point exponentially approach each other. The union of all unstable manifolds 



— u 

Figure 1: Unstable (u) and stable (s) manifolds associated with a phase point 
(thick bullet) in a two dimensional representation. 

of the phase points representing the system is called the unstable manifold 
in the system's phase space; the union of all stable manifolds of the phase 
points representing the system is called the stable manifold in the system's 
phase space. One now makes a (Markov) partition of the system's phase 
space into "parallelograms" (cells) (cf.fig.2). 

The "horizontal sides" of the parallelograms form together the unstable 
manifold and the "vertical" sides of the parallelograms form together the 
stable manifold in the phase space. The size of these parallelograms is de- 
termined by a parameter T, so that for T — ► their sizes go to zero. Now 
each cell Ej in phase space is given a weight A~\.(xj) which is equal to the 
inverse of the phase space expansion along the unstable manifold, during a 
time r. This phase space expansion is based on a trajectory moving dur- 
ing a (discrete) time r from — r/2 — 1 to r/2 along a phase space trajectory 
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Figure 2: Markov partition of a two dimensional phase space with cells Ej, 
formed by the unstable (u) and stable (s) manifolds of the phase points. 



through the center Xj of the cell Ej, using the dynamical equations of motion 
(cf.fig.3). 

Considering a small phase space volume A around the initial point at 
— t/2 — 1 (cf.fig.4), then all points in A will go via phase space trajectories to 
corresponding points in the phase space volume B around the final point at 
+r/2. The larger the phase space volume expansion A UiT (xj) in the direction 
of the unstable manifold u is, i.e. the larger Lb/ La, the more the phase 
space trajectories will tend to avoid (bypass) the point Xj. The inverse of 




Figure 3: Trajectory segment of duration r through the center Xj of cell Ej 
of a Markov partition of the system in phase space. 
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this ratio ~ La/ Lb ~ A 



U.T 1 



will therefore be a measure of the "eagerness'' 



or frequency of the phase space trajectories to be near Xj, i.e. that the system 
will visit the cell Ej. Weighing the Markov partitions in phase space this way, 
one obtains in a dynamical way the probability to find the system anywhere 
in phase space. As a consequence, the average of a smooth function F(x), 
where x denotes a point in phase space, is then determined by the SRB 
measure A.~\{xj): 



HSRB{dx)F(x) 



lim 

T>t/2^oo 



>:A,'.(.o)/-i 

A UT {xj : 



This appears to me a direct example of Einstein's proposal of a probability for 
complexions based on the system's dynamics rather than on ad hoc, although 
possibly reasonable, probabilistic assumptions as made by Boltzmann and 
Planck. 




Figure 4: Small phase space volume of extension La at time —t/2 — 1 and 
of extension Lb-, in the direction of the unstable manifold (u) at time t/2. 
Lb /La gives the phase space volume increase in the direction of the unstable 
manifold (w). 



VI. Final Remarks 

1. In thermal equilibrium the SRB distribution reduces to Gibbs' micro- 
canonical ensemble. This implies that this ensemble can also be derived 
rigorously dynamically along the lines sketched here, albeit so far only for 
smooth and sufficiently chaotic systems. The role of coarse graining phase 
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space into physically infinitesimal cells, to go from a microscopic to a macro- 
scopic description, which is used in various ways in all classical derivations 
of thermodynamics from statistical mechanics and also by Boltzmann and 
Planck, is only used in passing in the SRB measure, because of the ultimate 
limr>r/2 — >oo- 

2. To actually use Einstein's dynamical method to determine the probabil- 
ities of cells in phase space in practice, requires a solution of the equations 
of motion. The only way, so far, around this fundamental difficulty has been 
to replace the effects of dynamics again by suitable probability distributions 
as is done e.g. in superstatistics' 12_14 l This has been applied successfully to 
certain distribution functions in turbulent flows by using e.g. log-normal or 
X 2 -distributions. 

3. To the best of my knowledge there is no definition of entropy beyond 
the linear (near equilibrium) regime. This is in my opinion a fundamental 
open question, whose solution may be essential to make real progress into the 
far from equilibrium region. Gallavotti and I^ 15 ' have conjectured that the 
entropy in nonequilibrium stationary states "far" from equilibrium cannot be 
defined, just like the heat content of a body in equilibrium cannot be defined, 
since it depends on how this heat is used' 16 '. Similarly, entropy differences 
such as entropy production or entropy transfers could be meaningful also far 
from equilibrium, just like heat production or heat transfer are in equilibrium. 

4. In all fairness to Boltzmann, it should be said that he also proposed to 
determine the probability of a state or a complexion of a system in a region of 
phase space of this system in equilibrium, by assuming that this probability 
would be proportional to the time the system spends in it i.e. involving the 
dynamics of the system. But for some reason he did not develop this idea 
further. 

Finally it would be, in my opinion, a grave mistake to think that Boltz- 
mann would have been opposed to these new dynamical developments. As I 
said before, he had a strong disposition for dynamics and I think his prob- 
abilistic work was presumably inspired to get rid of his many opponents to 
his dynamical work and was just an intermezzo for him, important as it is 
for us. 

In fact, Boltzmann would have been delighted to see that a dynamical the- 
ory of phase space weights was developed and used a century after his death 
and that therefore his dynamical predilection was justified in the end^ 17 '. 
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